Apparatus and method for controlling operation of mobile terminal

ABSTRACT

A method of controlling the operation of a mobile terminal is provided. By dynamically selecting or setting a specific function of a mobile terminal according to a change in pressure or contact areas sensed by a plurality of touch sensors which cover the mobile terminal, a user can easily select or set a specific function of the mobile terminal according to his or her intention, and easily recognize information displayed on a screen of the mobile terminal.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of prior application Ser.No. 15/630,249, filed on Jun. 22, 2017, which has issued as U.S. Pat.No. 10,268,311 on Apr. 23, 2019 and is a continuation application ofprior application Ser. No. 14/922,431, filed on Oct. 26, 2015, which hasissued as U.S. Pat. No. 9,733,757 on Aug. 15, 2017 and is a continuationapplication of prior application Ser. No. 14/555,895, filed on Nov. 28,2014, which has issued as U.S. Pat. No. 9,170,698 on Oct. 27, 2015,which is a continuation application of prior application Ser. No.12/076,945, filed on Mar. 25, 2008, which has issued as U.S. Pat. No.8,928,594 on Jan. 6, 2015 and claimed the benefit under 35 U.S.C. §119(a) of a Korean patent application No. 10-2007-0092097, filed on Sep.11, 2007, the disclosure of which is incorporated herein in its entiretyby reference.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a method for controlling the operationof a mobile terminal, and more particularly, to an apparatus and methodfor controlling the operation of a mobile terminal using a sensor whichis disposed on the mobile terminal.

Description of the Related Art

Mobile terminals such as cellular phones have recently been developed tointegrate and merge the functions of various electronic products, inaddition to providing the functions of voice calls, and transmission andreception of text messages. For example, mobile phones which function asan MP3 player, a digital camera, an electronic dictionary, a digital TV,etc. are being developed.

Along with an increase in functions of a mobile terminal, extracomponents (for example, buttons) for executing the functions are added.However, when the number of buttons of a mobile terminal increases,users find it more difficult to rapidly select a button to execute adesired function or to rapidly set a desired function.

Meanwhile, instead of selecting a specific function by pressing a buttonof a mobile terminal, a method of displaying a Graphic User Interface(GUI) on a screen of the mobile terminal and allowing a user to select aspecific function through the GUI has been proposed. However, in themethod, since part of the screen of the mobile terminal should be usedto display the GUI, users may have difficulties in recognizing allinformation displayed on the screen of the mobile terminal. For example,if a GUI for zooming in or out on a map overlaps part of a screen onwhich the map is displayed, there may be difficulty in viewing the partof the screen that the GUI overlaps.

Therefore, instead of allowing a user to select or set a specificfunction of a mobile terminal using a physical button or through a GUI,a new user interface that selects or sets a specific function by sensinga user's motion is required.

SUMMARY OF THE INVENTION

The present invention provides an apparatus and method for selecting orsetting a specific function of a mobile terminal by sensing a user'smotion, instead of allowing the user to select or set the specificfunction by using a physical button or through a Graphic User Interface(GUI).

In order to achieve the object, the present invention discloses anapparatus and method for controlling the operation of a mobile terminal,by executing functions or applications of the mobile terminal,dynamically, according to information about pressure applied by a userto a touch sensor which is disposed on the mobile terminal.

According to an aspect of the present invention, there is provided anapparatus for controlling an operation of a mobile terminal, including:a plurality of touch sensors disposed on the mobile terminal; arecognition unit recognizing a function corresponding to pressureapplied to the mobile terminal and/or to contact areas of the mobileterminal which a user contacts, wherein the pressure and the contactareas are sensed by the plurality of touch sensors; and a control unitcontrolling an operation corresponding to the function.

The plurality of touch sensors are a plurality of pressure touch sensorswhich sense pressure applied to the outer surface of the mobileterminal, or a plurality of capacitive touch sensors whose capacitanceschange according to pressure applied to the outer surface of the mobileterminal. The plurality of touch sensors are disposed on at least one offront and rear surfaces of the mobile terminal.

The recognition unit includes: a storage unit storing information aboutfunctions corresponding to pressure applied to the mobile terminal orcontact areas of the mobile terminal which a user contacts; and afunction recognition unit searching for the function corresponding tothe pressure and/or the contact areas of the mobile terminal, sensed bythe plurality of touch sensors, from the storage unit, and providinginformation about the function to the control unit.

The function recognition unit recognizes the function corresponding tothe pressure and/or the contact areas of the mobile terminal, sensed bythe plurality of touch sensors, through any one of a neural network,template matching, a hidden Markov model, and a support vector machine(SVM).

According to another aspect of the present invention, there is provideda method for controlling an operation of a mobile terminal, including:sensing pressure applied by a user to the mobile terminal, and/orcontact areas of the mobile terminal which the user contacts;recognizing a function corresponding to the sensed pressure and/or thesensed contact areas of the mobile terminal; and controlling anoperation of the mobile terminal according to the function.

The sensing of the pressure and/or the contact areas of the mobileterminal includes: receiving a signal from a plurality of touch sensorsarranged on an outer surface of the mobile terminal; and sensing thepressure and/or the contact areas of the mobile terminal using thesignal received from the plurality of touch sensors.

Additional aspects of the invention will be set forth in the descriptionwhich follows, and in part will be apparent from the description, or maybe learned by practice of the invention.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate exemplary embodiments of theinvention, and together with the description serve to explain theaspects of the invention.

FIG. 1 is a block diagram of an apparatus for controlling the operationof a mobile terminal, according to an embodiment of the presentinvention;

FIG. 2 is a cross-sectional view of a mobile terminal according to anembodiment of the present invention;

FIGS. 3A and 3B illustrate arrangements of touch sensors which cover themobile terminal illustrated in FIG. 2, according to an embodiment of thepresent invention;

FIGS. 4A, 4B, and 4C are views showing contact areas that a user'sfingers contact at the front and rear surfaces of a mobile terminal, anda picture in which the user grips the mobile terminal, according to anembodiment of the present invention;

FIGS. 5A through 5F are views showing contact areas that a user'sfingers contact at the front and rear surfaces of a mobile terminal, andpressure applied to the contact areas of the front and rear surfaces ofthe mobile terminal, and pictures in which the user grips the mobileterminal, according to an embodiment of the present invention;

FIG. 6 is a flowchart of a method for controlling the operation of amobile terminal, according to an embodiment of the present invention;

FIG. 7 illustrates an arrangement of 8 touch sensors which cover themobile terminal;

FIG. 8 is a flowchart of a method for controlling the operation of amobile terminal, according to another embodiment of the presentinvention; and

FIGS. 9A and 9B illustrate changes in contact areas of the touch sensorsillustrated in FIG. 7 or in pressure applied to the contact areas, whenthe mobile terminal operation control method illustrated in FIG. 8 isexecuted.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The invention is described more fully hereinafter with reference to theaccompanying drawings, in which exemplary embodiments of the inventionare shown. This invention may, however, be embodied in many differentforms and should not be construed as limited to the exemplaryembodiments set forth herein. Rather, these exemplary embodiments areprovided so that this disclosure is thorough, and will fully convey thescope of the invention to those skilled in the art. In the drawings, thesize and relative sizes of layers and regions may be exaggerated forclarity. Like reference numerals in the drawings denote like elements.

FIG. 1 is a block diagram of an apparatus for controlling the operationof a mobile terminal, according to an embodiment of the presentinvention.

Referring to FIG. 1, the mobile terminal operation control apparatusincludes a sensor unit 10, a signal processor 11, a recognition unit 12,a storage unit 13, a controller 14, and a user interface 15.

The sensor unit 10 is disposed on the outer surface of the mobileterminal, and includes a plurality of touch sensors. The touch sensorsmay be pressure touch sensors which sense pressure applied to the outersurface of the mobile terminal, or capacitive touch sensors whosecapacitances change according to pressure applied to the outer surfaceof the mobile terminal.

FIG. 2 is a cross-sectional view of a mobile terminal according to anembodiment of the present invention.

As illustrated in FIG. 2, it is preferable that touch sensors 20 and 22are respectively arranged on the front and rear surfaces of a mobileterminal body 21. Also, a plurality of touch sensors can be disposed onat least one lateral surface of the mobile terminal body 21.

Exemplary arrangements of the touch sensors that cover the front andrear surfaces of the mobile terminal are shown in FIGS. 3A and 3B.

As illustrated in FIG. 3A, a transparent touch sensor is disposed on anarea corresponding to a display screen such as a Liquid Crystal Display(LCD), and an opaque touch sensor is disposed on the remaining area, inthe front surface of the mobile terminal.

Also, an opaque touch sensor is disposed on the rear surface of themobile terminal. The opaque touch sensor disposed on the rear surface ofthe mobile terminal may be a flexible opaque touch sensor.

Meanwhile, in an arrangement of touch sensors illustrated in FIG. 3B, atransparent touch sensor is disposed on the entire front surface of amobile terminal, and an opaque touch sensor is disposed on the entirerear surface of the mobile terminal. The opaque touch sensor disposed onthe rear surface of the mobile terminal may be a flexible opaque touchsensor.

Returning to FIG. 1, the signal processor 11 recognizes whether themobile terminal is gripped by a user through the plurality of touchsensors. If the mobile terminal is gripped by the user the signalprocessor 11 receives information about contact areas of the mobileterminal that the user's fingers contact and about pressure applied tothe contact areas, which are sensed by the plurality of touch sensors,and transmits the information about the contact areas or pressure to therecognition unit 12.

The recognition unit 12 recognizes a function corresponding to theinformation about the contact areas or pressure, on the basis of datastored in the storage unit 15, thereby recognizing the user's intention.That is, the storage unit 15 stores functions corresponding toinformation about contact areas or pressure.

For example, the signal processor 11 determines whether the mobileterminal is grasped by a user, through the sensor unit 10, asillustrated in FIGS. 4A and 4B. If it is determined that the mobileterminal is gripped by the user the signal processor 11 then detectscontact areas of the mobile terminal which the user contacts, accordingto a change in pressure applied to the sensor unit 10 or in capacitanceof the sensor unit 10, as illustrated in FIGS. 5A and 5B or in FIGS. 5Cand 5D, and provides information about the contact areas to therecognition unit 12.

FIG. 4A is a view showing contact areas that a user's fingers contact atthe front surface of a mobile terminal, wherein the contact areas areshadowed.

FIG. 4B is a view showing contact areas that the user's fingers contactat the rear surface of the mobile terminal, wherein the contact areasalso are shadowed.

FIG. 4C illustrates a picture in which the user just grips the mobileterminal without applying any pressure to the mobile terminal.

FIG. 5A is a view showing contact areas that the user's fingers contacton the front surface of the mobile terminal, and pressure applied to thecontact areas, when the user bends the mobile terminal inward, whereinthe contact areas are shadowed. Comparing FIG. 5A to 4A, the contactareas having the same shadow level as illustrated in FIG. 4A areconverted into first, second, and third contact areas 50, 51, and 52having different shadow levels as illustrated in FIG. 5A. In FIG. 5A,the order of the pressure values of the first, second, and third areas50, 51, and 52 is the first area 50>the second area 51>the third area52.

FIG. 5B is a view showing contact areas that the user's fingers contactat the rear surface of the mobile terminal, and pressure applied to thecontact areas, when the user bends the mobile terminal inward, whereinthe contact areas are shadowed. Comparing FIG. 5B to 4B, the contactareas having the same shadow level as illustrated in FIG. 4B areconverted into fourth, fifth, and sixth contact areas 53, 54, and 55having different shadow levels as illustrated in FIG. 5B. In FIG. 5B,the order of the pressure values of the fourth, fifth, and sixth areas53, 54, and 55 is the fourth area 53>the fifth area 54>the sixth area55.

FIG. 5C illustrates a picture in which the user grips the mobileterminal while bending the mobile terminal inward.

FIG. 5D is a view showing contact areas that the user's fingers contacton the front surface of the mobile terminal, and pressure applied to thecontact areas, when the user bends the mobile terminal outward, whereinthe contact areas are shadowed. Comparing FIG. 5D to FIG. 4A, thecontact areas having the same shadow level as illustrated in FIG. 4A areconverted into seventh, eighth, and ninth contact areas 56, 57, and 58having different shadow levels as illustrated in FIG. 5D. In FIG. 5D,the order of the pressure values of the seventh, eighth, and ninth areas56, 57, and 58 is the seventh area 56>the eighth area 57>the ninth area58.

FIG. 5E is a view showing contact areas that the user's fingers contacton the rear surface of the mobile terminal, and pressure applied to thecontact areas, when the user bends the mobile terminal outward, whereinthe contact areas are shadowed. Comparing FIG. 5E to FIG. 4B, thecontact areas having the same shadow level as illustrated in FIG. 4B areconverted into tenth, eleventh, and twelfth contact areas 59, 60, and 61having different shadow levels as illustrated in FIG. 5E. In FIG. 5E,the order of the pressure values of the tenth, eleventh, and twelfthcontact areas 59, 60, and 61 is the tenth area 59>the eleventh area60>the twelfth area 61.

FIG. 5F illustrates a picture in which the user grips the mobileterminal while bending the mobile terminal outward.

As described above, the recognition unit 12 recognizes the user'sintention on the basis of function information corresponding toinformation about pressure or contact areas, which is stored in thestorage unit 13.

The recognition unit 12 can perform pattern recognition in order torecognize the user's intention, on the basis of the information aboutpressure and contact areas.

General pattern recognition methods will be described below.

First, a large amount of data about an input X and a class C iscollected from a user. Second, the collected data is classified intolearning data and test data. Third, the learning data is provided to apattern recognition system to perform learning. Then, model parametersof the pattern recognition system are changed according to the learningdata. Finally, only the input X is provided to the pattern recognitionsystem, so that the pattern recognition system outputs the class C.

A pattern recognition method according to a first embodiment of thepresent invention uses a neural network. A neural network, which is amodel for simplifying a synaptic neurotransmission process of livingthings and mathematically interpreting the synaptic neurotransmissionprocess, is used to analyze problems through a kind of learning processfor adjusting the connection weights of neurons which are interconnectedin a complex pattern while passing data through the neurons. The patternrecognition method is similar to the learning and memorizing process ofthe human brain, and deduction, classification, prediction, etc. can beperformed through the pattern recognition method. In the neural network,a neuron corresponds to a node, and a connection strength betweenneurons corresponds to a weight connected to an arc between nodes. Theneural network may be a multi-layer perceptron neural network consistingof a plurality of single-layer perceptrons.

In a pattern recognition method according to a second embodiment of thepresent invention a user's motion pattern is recognized using a supportvector machine (SVM). According to the pattern recognition method, aN-dimensional space is formed using characteristics of N motion signals,a proper hyperplane is created from the N-dimensional space usinglearning data, and a pattern is recognized from the hyperplane.

In a pattern recognition method according to a third embodiment of thepresent invention a motion pattern is recognized using templatematching. According to the pattern recognition method, patterns areclassified on the basis of learning data to create template data,template data which is most similar in distance to an input value issearched for from among the template data, and a pattern is recognizedfrom the search template data.

A pattern recognition method according to a fourth embodiment of thepresent invention uses a hidden Markov model. The hidden Markov model isa finite state machine having transition probabilities between states,wherein each state cannot be directly observed, but can be recognizedfrom observable states.

Returning to FIG. 1, the controller 13 generates an operation controlsignal regarding the function recognized by the recognition unit 12, andtransfers the operation control signal to the user interface unit 14.

The user interface unit 14 performs a specific function of the mobileterminal or an application operation which is being executed, accordingto the operation control signal.

For example, if the user bends the mobile terminal inward, while a webpage is displayed and executed on the screen of the mobile terminal, theweb page can be reduced.

FIG. 6 is a flowchart of a method for controlling the operation of amobile terminal, according to an embodiment of the present invention.

As illustrated in FIG. 6, it is determined whether the mobile terminalis contacted by a user, through a plurality of touch sensors which coverthe outer surface of the mobile terminal (operation S60). That is, it isdetermined whether the mobile terminal is gripped by a user, through aplurality of touch sensors.

At this point, the plurality of touch sensors may be a plurality ofpressure touch sensors which sense pressure applied to the outer surfaceof the mobile terminal, or a plurality of capacitive touch sensors whosecapacitances change according to pressure applied to the outer surfaceof the mobile terminal.

If the plurality of touch sensors determine that the mobile terminal isgripped by the user, pressure applied to the mobile terminal orcapacitance of the plurality of touch sensors is measured (operationS61). Then, it is determined whether a change in pressure or a change incontact areas due to a change in capacitance occurs according to themeasurement result of the pressure or capacitance (operation S62).

When the change in pressure or the change in contact areas due to thechange in capacitance is detected, a function corresponding to thechanged pressure or the changed contact areas is searched for in adatabase, or a function corresponding to information about the changedpressure or the changed contact areas is recognized through patternrecognition (operation S63).

The pattern recognition can be performed through a neural network,template matching, a hidden Markov model, a support vector machine(SVM), etc.

Then, the mobile terminal generates an operation control signalcorresponding to the function to control the function or the operationof an application which is being executed (operation S64).

Hereinafter, a method for controlling the operation of a mobileterminal, according to an embodiment of the present invention, will bedescribed by using as an example a case where 8 touch sensors aredisposed on the mobile terminal. Here, the number of touch sensors isnot limited to 8.

An arrangement of the 8 touch sensors which cover the mobile terminal isillustrated in FIG. 7.

Referring to FIG. 7, touch sensors 1, 2, 3, and 4 are arranged on thefront surface of the mobile terminal in such a manner that the touchsensors 1 and 2 and the touch sensors 3 and 4 are arranged with adisplay screen of the mobile terminal in between, and touch sensors 5,6, 7, and 8 are arranged at regular intervals on the rear surface of themobile terminal.

FIG. 8 is a flowchart of a method for controlling the operation of amobile terminal according to another embodiment of the presentinvention.

In the current embodiment, it is assumed that the mobile terminal hasthe arrangement of 8 touch sensors as illustrated in FIG. 7.

Referring to FIGS. 7 and 8, it is determined whether the mobile terminalis contacted by a user, through 8 touch sensors 1 through 8 which arearranged on the outer surface of the mobile terminal (operation S80).That is, it is determined whether the mobile terminal is gripped by auser, through the 8 touch sensors 1 through 8.

At this point, the 8 touch sensors 1 through 8 may be pressure touchsensors which sense pressure applied to the outer surface of the mobileterminal, or capacitive touch sensors whose capacitances changeaccording to pressure applied to the outer surface of the mobileterminal. If it is determined through the 8 touch sensors 1 through 8that the mobile terminal is gripped by a user, pressure applied to themobile terminal or capacitance of the touch sensors 1 through 8 ismeasured through the 8 touch sensors 1 through 8 (operation S81). Then,it is determined whether a change in pressure or a change in contactareas due to a change in capacitance occurs according to the measurementresult of the pressure or capacitance (operation S82). For example, itis determined whether values sensed by the touch sensors 2, 3, 5, and 8are above a predetermined value, and values sensed by the touch sensors1, 4, 6, and 7 are below the predetermined value.

If the values sensed by the touch sensors 2, 3, 5, and 8 are above thepredetermined value, and the values sensed by the touch sensors 1, 4, 6,and 7 are below the predetermined value, the mobile terminal determinesthat the user bends the mobile terminal inward, and controls anoperation corresponding to the inward manipulation (operation S83).

When the user bends the mobile terminal inward, changes in the states ofthe touch sensors 1 through 8 of the mobile terminal are illustrated inFIG. 9A.

In FIG. 9A, the touch sensors 2 and 3, among the touch sensors 1, 2, 3,and 4 arranged on the front surface of the mobile terminal, areillustrated by a dark color in order to represent that pressure orcapacitance values sensed by the touch sensors 2 and 3 are above thepredetermined value. Also, the touch sensors 5 and 8, among the touchsensors 5, 6, 7, and 8 arranged on the rear surface of the mobileterminal, are illustrated by a dark color in order to represent thatpressure or capacitance values sensed by the touch sensors 5 and 8 areabove the predetermined value.

Meanwhile, if it is determined in operation S83 that the pressure orcapacitance values sensed by the touch sensors 2, 3, 5, and 8 are notabove the predetermined value, and the pressure or capacitance valuessensed by the touch sensors 1, 4, 6, and 7 are not below thepredetermined value, it is determined whether the pressure orcapacitance values sensed by the touch sensors 2, 3, 5, and 8 are belowthe predetermined value, and the pressure or capacitance values sensedby the touch sensors 1, 4, 6, and 7 are above the predetermined value(operation S84).

If the pressure or capacitance values sensed by the touch sensors 2, 3,5, and 8 are below the predetermined value, and the pressure orcapacitance values sensed by the touch sensors 1, 4, 6, and 7 are abovethe predetermined value, the mobile terminal determines that the userbends the mobile terminal outward, and controls an operationcorresponding to the outward manipulation (operation S85).

When the user bends the mobile terminal outward, changes in the statesof the touch sensors 1 through 8 of the mobile terminal are illustratedin FIG. 9B.

In FIG. 9B, the touch sensors 1 and 4, among the touch sensors 1, 2, 3,and 4 arranged on the front surface of the mobile terminal, areillustrated by a dark color in order to represent that pressure orcapacitance values sensed by the touch sensors 1 and 4 are above thepredetermined value. Also, the touch sensors 6 and 7, among the touchsensors 5, 6, 7, and 8 arranged on the rear surface of the mobileterminal, are illustrated by a dark color in order to represent thatpressure or capacitance values sensed by the touch sensors 6 and 7 areabove the predetermined value.

As described above, in an apparatus and method for controlling theoperation of a mobile terminal, according to the present invention, byproviding a new user interface for performing a specific functionaccording to the user's intention recognized by sensing his or hermanipulation, the user can easily select or set a specific function ofthe mobile terminal and easily recognize information which is displayedon a screen of the mobile terminal.

That is, by dynamically selecting or setting a specific function of amobile terminal according to a change in pressure or contact areassensed by a plurality of touch sensors which cover the mobile terminal,a user can easily select or set a specific function of the mobileterminal according to his or her intention, and easily recognizeinformation which is displayed on a screen of the mobile terminal.

It will be apparent to those skilled in the art that variousmodifications and variations can be made in the present inventionwithout departing from the spirit or scope of the invention. Thus, it isintended that the present invention covers the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents.

What is claimed is:
 1. A mobile phone, comprising: a display; aplurality of touch sensors configured to sense a plurality of pressurevalues of a user input on respective portions of the display; and atleast one processor configured to: recognize a function from among aplurality of functions of the mobile phone, the function being afunction corresponding to at least one pressure value, from among thesensed plurality of pressure values, sensed by at least one of theplurality of touch sensors, and operate the recognized function.
 2. Themobile phone of claim 1, wherein the plurality of touch sensors are oneof a plurality of pressure touch sensors, which sense pressure appliedto an outer surface portion of the mobile phone, or a plurality ofcapacitive touch sensors, capacitances of which change according topressure applied to the outer surface portion of the mobile phone. 3.The mobile phone of claim 1, wherein the respective portions comprise afront surface portion of the mobile phone and a rear surface portion ofthe mobile phone.
 4. The mobile phone of claim 1, further comprising: astorage configured to store information about the plurality offunctions, wherein the plurality of functions corresponding to aplurality of pressure value patterns, respectively, each of theplurality of pressure value patterns being a pattern of a set of atleast one pressure value due to pressure to be collectively applied onthe mobile phone, and wherein the at least one processor is furtherconfigured to: search for information about the function correspondingto the plurality of sensed pressure values in the storage, and providefound information about the function to the controller.
 5. The mobilephone of claim 1, wherein the at least one processor is furtherconfigured to recognize the function from among the plurality offunctions based on a determination as to whether a pattern of theplurality of sensed pressure values indicates that the mobile phone isbeing bent.
 6. The mobile phone of claim 5, further comprising: a userinterface configured to display a page, wherein the mobile phone isconfigured to control the page displayed by the user interface to bereduced based on a result of the determination.
 7. The mobile phone ofclaim 5, wherein the mobile phone is configured to determine whether themobile phone is bent inward or outward based on an ordering of at leastsome of the plurality of sensed pressure values.